Deep Multi-Modal Classification of Intraductal Papillary Mucinous Neoplasms (IPMN) with Canonical Correlation Analysis

摘要

Pancreatic cancer has the poorest prognosis among all cancer types.Intraductal Papillary Mucinous Neoplasms (IPMNs) are radiographicallyidentifiable precursors to pancreatic cancer; hence, early detection andprecise risk assessment of IPMN are vital. In this work, we propose aConvolutional Neural Network (CNN) based computer aided diagnosis (CAD) systemto perform IPMN diagnosis and risk assessment by utilizing multi-modal MRI. Inour proposed approach, we use minimum and maximum intensity projections to easethe annotation variations among different slices and type of MRIs. Then, wepresent a CNN to obtain deep feature representation corresponding to each MRImodality (T1-weighted and T2-weighted). At the final step, we employ canonicalcorrelation analysis (CCA) to perform a fusion operation at the feature level,leading to discriminative canonical correlation features. Extracted featuresare used for classification. Our results indicate significant improvements overother potential approaches to solve this important problem. The proposedapproach doesn't require explicit sample balancing in cases of imbalancebetween positive and negative examples. To the best of our knowledge, our studyis the first to automatically diagnose IPMN using multi-modal MRI.